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This Positive Behavior Interventions and Supports (PBIS) school status report provides a summary of the current status 
of all schools implementing PBIS in Minnesota, including participation in the Minnesota Department of Education’s 
training sequence, implementation fidelity assessment scores, and outcomes. In particular, this report highlights the 
patterns and trends in PBIS implementation and outcomes in Minnesota, and helps to understand the relationships 
between training, implementation fidelity, and outcomes. This report is intended for all stakeholders to better understand 
the status of PBIS in Minnesota. 


About PBIS Schools 

Currently, there are 651 schools in Minnesota that have been ee ra 
trained or are in training funded by the Minnesota Department @ north 

of Education (MDE) to implement PBIS. The tables below @ south 


illustrate the number of schools by region, grade level, and © Metro 
training cohort. 


Number of trained schools (by region) 


Region Number _ Percent 

Metro ae = Number of schools trained in each cohort 

North 187 29% 

South 176 7% Cohort Number 

Total 651 100% 1 (2005-2007) 9 
2 (2006-2008) 11 

Number of trained schools (by grade level) 3 (2007-2009) 42 

Grades served Number _ Percent 4 (2008-2010) 32 

Elementary (Grades PK-6) a7 58% 5 (2009-2011) 47 

Middle school (Grades 5-8) 94 14% 6 (2010-2012) 82 

Junior High (Grades 7-8 or 7-9) 7 1% 7 (2011-2013) 81 

Senior High (Grades 9-12) 53 8% 8 (2012-2014) 82 

Secondary School (Grades 7-12) 59 9% 9 (2013-2015) 57 

Combined School (Grades K-12) 4 1% 10 (2014-2016) 57 

ALC/ALP 24 4% 11 (2015-2017) 57 

Special Education 26 4% 12 (2016-2018) 51 

Other/Unknown* 7 1% 13 (2017-2019) 58 

Total 651 100%* Total 651 

*Other categories include correctional programs, residential care, and day Notes. Total number includes 11 school teams in Cohorts 12 and 13 from a 

treatment programs. metro school district that participated in a modified cohort training schedule. 
Schools will not sum to 651 due to 15 schools that were involved in multiple 
cohorts. Schools are counted in each cohort in which they were trained. 
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Cohorts 1-13 


During the 2017-18 school year, 71 schools that were 
not a part of any Minnesota PBIS sponsored training 
cohort submitted implementation fidelity data (TFI, 
BoQ, and/or a SET) at least one time. Although these 
schools have not been formally trained to use PBIS, 
they are still eligible to apply for recognition school 
status. The number of non-cohort PBIS schools by 
region is outlined to the right. 


Training participation 


For schools that participated in the two-year, MDE- 
sponsored PBIS training sequence during the 2017-18 
school year (Cohort 12 and 13 schools), three indicators 
are used to illustrate how fully schools are participating 
and invested in the training: team attendance at training, 
administrator attendance at training, and ratings of 
training usefulness. 


Team and administrator attendance at training 


Schools in the first year of training in 2017-18 (Cohort 
13) had six full days of training that they were expected 
to attend. Schools in the second year of training in 2017- 
18 (Cohort 12) had six days of training in their first year 
and three full days of training in the second year for a total 
of nine days throughout their two-year training sequence. 


Support from administration is a key feature of schools 
that successfully implement PBIS. Schools that are 
participating in the two-year training sequence are 
expected to bring an administrator (principal, vice- 
principal, or assistant principal) to training. 


Schools and administrators in both cohorts had high 
rates of training attendance. In Cohort 12, team 
attendance in the north region was lower compared to 
schools in other regions while administrators in the 
south region in Cohort 12 had the highest attendance 
rate. Cohort 12 administrators in the south region 
attended more than eight full days of training, on 
average. In Cohort 13, team attendance was roughly 
equal across regions, while administrator attendance 
was somewhat lower in the metro and north regions 
than in the south. 


Number of non-cohort schools that submitted implementation 
fidelity data in 2017-2018 school year (by region) 


Region Number _ Percent 
Metro 38 54% 
North 6 8% 
South 27 38% 
Total 71 100% 


Average number of training days by team and administrator 
(Cohort 12 - Year 2 in training sequence, max. = 9.0 days) 


9.0 8.9 
8.2 8.3 8.7 


Metro North South State average 
(N=18) (N=13) (N=8) (N=39) 
aTeam = Administrator 


Note. N’s in this chart will not match the total number of schools in the cohort. 
Nine schools from a metro school district follow a modified training schedule 
and have thus been excluded. Two schools from a south region school district 
with incomplete data were also excluded 


Average number of training days by team and administrator 
(Cohort 13- Year 1 in training sequence, max. = 6.0 days) 


5.7 5.6 595, 58 


45 5.0 5.1 


Metro North South State average 
(N=13) (N=8) (N=28) (N=49) 
= Team = Administrator 


Note. N’s in this chart will not match the total number of schools in the 
cohort. Six schools from the north region and metro region followed 
modified training schedules and have thus been excluded. A district from 
the north region had incomplete data so this was also excluded. 
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Usefulness of training 


School teams were asked to provide feedback on the training sessions they attended. Participants were asked to rate the 
level of usefulness of training by answering the following question: “Overall, how useful was this PBIS training in terms 
of helping your team to implement PBIS in your school?” Overall, school teams rated the trainings to be highly useful. With 
only one exception, 95% or more respondents rated the trainings to be either very or somewhat useful. Cohort 13 in the 
North region had 100% of schools rate training as either very or somewhat useful. The one exception noted above, Cohort 
12 in the North region, still had 88% of schools rate training as either very or somewhat useful. The graph below illustrates 


the averaged usefulness ratings from the fall, winter, and spring trainings for the 2017-18 school year. 


“Overall, how useful was this PBIS training in terms of helping your team to implement PBIS in your school?” 


(Combined results, all trainings in 2017-18). 
Cohort 12 


39% 


56% 


= Somewhat useful 


3% — 4% 
Metro North South 
(N=66) (N=26) (N=25) 


= Very useful 


= Not very useful 


Not at all useful 


Cohort 13 


67% 
83% 


, 5% 


f T — 5% 
Metro North South 
(N=40) (N=28) (N=72) 


Note: The total N is the overall number of school teams from each region that answered the question “Overall, how useful was this PBIS training in terms of 
helping your team implement PBIS in your school?” combined for the fall, winter, and spring training surveys. This will differ from the total number of school 


teams from each region in Cohorts 12 and 13. 


Implementation fidelity measures 


PBIS is an evidence-based framework for preventing 
problem behavior, providing instruction and support for 
positive and prosocial behaviors, and supporting social, 
emotional, and behavioral needs for all students. A 
significant amount of research has been done to identify 
the critical features of PBIS. In general, implementation 
science points to a specific sequence to ensure the model 
is implemented with fidelity. The following assessment 
tools are currently used to assess PBIS implementation 
fidelity among participating schools in Minnesota: the 
Tiered Fidelity Inventory (TFI), Benchmarks of Quality 
(BoQ), and the Self-Assessment Survey (SAS). In the 
2015-16 school year, the TFI replaced two other 
assessments: the School-wide Evaluation Tool (SET), 
and the Team Implementation Checklist (TIC) which were 
used to measure implementation fidelity as part of the 
cohort scope and sequence in Minnesota. Therefore, in 
lieu of the SET and the TIC, school teams in training 
are trained and coached to use the TFI to action plan 


during training and so they can sustain PBIS 
implementation beyond training by using the TFI at least 
annually to measure fidelity and develop an action plan 
for priority items that are not fully in place. More 
information about each of the assessments follows below. 


TFI scores 


The Tiered Fidelity Inventory, or TFI, is used to measure 
implementation fidelity across all three tiers of PBIS. 
However, schools participating in PBIS cohort training 
are trained and coached to complete Tier 1 of the TFI. 
Schools in training complete the TFI at four points during 
the school year: once at each cohort training session and 
once in the spring outside of training with the support of 
an external TFI facilitator. At that time, the external 
facilitator completes a walkthrough assessment of the 
school before facilitating the TFI with the school’s PBIS 
team. A school is considered to be implementing 
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school-wide PBIS with fidelity if they have an overall 
Tier | scale score of 70 or higher. Below are the average 
TFI scores for Tier 1 for Cohort 12 (schools in year two 
of training) and Cohort 13 (schools in year one of 
training). Over time, TFI scores for school teams in both 
cohorts rose steadily. 


Average TFI scores for Cohort 12 
(Schools in year 2 of training sequence) 


Year 1 (2016-2017) Year 2 (2017-2018) 


78% 79% 


46% 
36% 
19%.-+ 


0% 
ii I ; 
= ul i 


Fall Winter i Endof Fall Winter Spring End of 
(n=39) (n=45) (n=46) Year (n=43) (n=45) (n=40) Year 
(n=39) (n=31) 


Average TFI scores for Cohort 13 
(Schools in year 1 of training sequence) 


Year 1 (2017-2018) 


Fall Winter Spring End of 
(n=51) (n=56) (n=51) Year 
(n=38) 
It is recommended by the State Leadership Team (SLT) 
that schools out of training should complete a TFI three 
times per year to measure progress and implementation 
fidelity. The SLT also recommends that these be 
completed by an external facilitator and a walkthrough 
be completed at least once per year. Out of 543 Cohort 
1-11 PBIS sustaining schools, only four completed the 
TFI at least three times during the 2017-18 school year. 
See figure below. 


Number of TFis completed by schools in Cohorts 1-11 during 
the 2017-18 School Year 


125 
68 
i 4 


1 TFl 2 TFls 3 or more TFIs 


Statewide, average scores on the most recently completed 
TFI for Cohort 1-11 school teams were just over the 70 
percent benchmark of implementation fidelity (72%). 
Scores in the South region were somewhat lower compared 
to the Metro and North regions. Meanwhile, the Metro 
region accounted for more than half (53%) of Cohort 1-11 
schools completing at least one TFI in the 2017-18 school 
year. See figure below. 


Average Tier 1 Score of Most Recently Completed TFI in the 
2017-18 School Year - Cohorts 1-11 (Schools out of training) 


72% 74% 75% 


; 63% 
Overall Metro North South 
(n=197) (n=104) (n=56) (n=37) 


Self-Assessment Surveys 


The Self-Assessment Survey, or SAS, examines the status 
and need for improvement of four behavior support systems: 
school-wide, non-classroom, classroom, and individual 
students. This is a survey of all school staff that helps to 
identify staff priorities for action planning, especially 
from staff members who are not currently on the PBIS 
team. It measures four of the five effective behavior 
support systems in schools that contribute to a positive 
school climate. A summary of the SAS results should be 
used to develop an action plan for implementing and 
sustaining PBIS systems throughout the school. Teams in 
training received support to administer and use the SAS 
twice during their first year and once during their second 
year. Schools out of training should complete the SAS on 
a yearly basis. 


During the 2017-18 school year, most school teams (83%) 
that were out of training did not complete a SAS 
assessment. In Cohort 12, 3 out of 10 teams did not 
complete a SAS assessment. In Cohort 13, about 1 out 
of 3 teams (35%) completed less than the recommended 
two SAS assessments. See figure below. 
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Number of SAS completed during last school year 
(by cohort) 


1% 


8% 
<= 


Cohorts 1-11 
(n=543) 


mOSAS #1SAS =2+SAS 


Cohort 12 
(N=50) 


Cohort 13 
(N=60) 


Benchmarks of Quality 


Schools that are out of training may choose to complete 
the Benchmarks of Quality, or BoQ, to assess their Tier 1 
PBIS implementation fidelity. The BoQ is a self- 
assessment tool that is taken by school teams. If a school 
team chooses to complete this fidelity assessment, it is 
recommended they complete a BoQ once a year in the 
spring. All school teams out of training are eligible to take 
the BoQ. Every school team that completed the BoQ in 
the 2017-18 school year (N=20) was implementing PBIS 
with fidelity (as indicated by a score of 70% or higher). 


BoQ score of schools in Cohorts 1-11 that completed during 
the 2017-18 school year (N=20) 


45% 


40% 


60% orless 60-69% 70-79% 80-89% 90% or 
(N=0) (N=0) (N=3) (N=9) greater 
(N=8) 

SET scores 


The School-wide Evaluation Tool, or SET, is used to 
measure implementation fidelity of Tier 1 of PBIS. A 
school is considered to be implementing PBIS with 
fidelity if they score an average of 80 or higher on their 
post-training SET. Although most schools are choosing 
to complete the TFI out of training, a few schools (n=7) 
completed the SET to measure implementation fidelity. 
Six of the seven schools had an overall score of 80 or 
above, indicating that these schools are implementing 
PBIS with fidelity. 


Behavioral data system 


Schools are expected to have student office discipline 
referral (ODR) decision system in place by the winter 
of their first year of training. The presence of a behavioral 
decision support data system for student outcomes allows 
school PBIS teams to precision problem solve around 
student outcomes to improve school climate and increase 
equity. The data collected in these systems are used for 
improving school-wide behavior support. One specific 
type of data tracked in these systems is office discipline 
referrals (ODRs). 


Of schools in Cohorts 1-11, more than half (57%) use the 
School-Wide Information System (SWIS), while one-third 
(33%) use other systems. In both Cohorts 12 and 13 at 
least half of schools’ behavioral systems are unknown. 
This is likely a result of a change in how this information 
is collected. Since the creation of the MN SW-PBIS 
database managed by Wilder Research, behavioral system 
data is collected based on ODR data stored in or submitted 
through that database. If a school does not submit their 
ODR data through either SWIS or the PBIS database, their 
behavioral system will be listed as unknown. Of those 
schools that did submit ODR data through SWIS or the 
PBIS database, SWIS was the most common behavioral 
system in both Cohorts 12 and 13 (40% and 25% of all 
school teams, respectively). See figure below. 


Behavioral data system used (by cohort) 


13% 
1% 
2% by 
: 
3% 
Cohorts 1-11 Cohort 12 Cohort 13 
(n=543) (n=50) (n=60) 


Unknown/No data system 
Non-SWIS (multiple systems) 
= Non-SWIS (one system) 
= SWIS only 
= SWIS and another system 


Note: Non-SWIS systems include: TIES, Infinite Campus, CLASSROOM, 
SKYWARD, Discovery, and custom systems. 
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Outcomes — Triangle data 


One goal of PBIS is to increase the efficiency with which schools respond to students’ behavioral support needs (Sailor, 
Dunlap, Sugai, & Horner, 2009). This efficiency can be maximized by minimizing the number of students who need 
support beyond the universal level. The hallmark conceptual triangle of PBIS suggests that, when PBIS is implemented 
effectively, on average, 80-90% of students are socially successful with universal support alone, 5-15% require 
additional support in the form of targeted interventions, and 1-5% require intensive individualized support (Sugai, 
Horner, & Gresham 2002). Based on this, Minnesota schools that have been trained to implement PBIS are, on 
average, within these targets. 


The smaller the percentage of students who require support beyond the universal level, the more efficiently those 
students can be served with the limited resources schools have available. “Socially successful” is defined as having 
0-1 office discipline referrals (ODR) within a given time span. “Needing targeted interventions” is defined as having 
2-5 ODR, and “needing intensive interventions” is defined as having more than 6 ODR within a given time span 
(Horner, Sugai, Todd, & Lewis-Palmer, 2005). 


Data are reported for schools in cohorts 1-13 that have entered data through one of two methods: through SWIS or 
through the PBIS database managed by Wilder Research. Results were combined and averaged statewide and for each 
region. The triangle scores are calculated using “Majors” only. Major behaviors are defined as disciplinary incidents 
that must be handled by administration. These may include, but are not limited to: physical fights, property damage, 
drugs, weapons, tobacco, etc. 


Overall, and in the Metro and South regions, school teams that reported ODR data are meeting or exceeding the target 
of 80-15-5 set forth by the Minnesota SLT. In the North region, school teams reported a slightly elevated percentage 
(7%) of students at the Tier 3 level. See figures below. 


Triangle data (Statewide, N=227) —_—‘ Triangle data (Metro, N=109) Triangle data (North, N=63) Triangle data (South, N=55) 


—_13% 
6% 


—_! 5% —14% 
8% 7% 


—17% 
10% 


M 6o0rmore ODRs \ 2to5 ODRs B® O0to1ODR 
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Issues to consider 


School participation. Many school teams (n=254, or 47%) that are out of training in Cohorts 1-11 did not submit 
either implementation fidelity assessment scores (TFI, SAS, BoQ, or SET) or ODR discipline data. RIPs should 
continue inviting these schools to reconnect with PBIS and offer incentives such as coaching, sustainability training, 
free TFI or BoQ facilitation, support with data systems, etc., to encourage this behavior. Research has shown that 
using data about their implementation of PBIS as well as about their outcomes (behavior data) is the most critical 
factor in determining PBIS sustainability at the school level.! 


Change benchmark for Year 2 Tiered Fidelity Inventory (TFD scores. Because the Minnesota PBIS Statewide 
Leadership Team wants to convey a focus on continuous improvement in implementation, we agreed at the July 2018 
SLT meeting to change the TFI benchmarks for schools in their second year of training to 65 at the fall training, 70 
at winter training, and 75 at spring training. (This increases the expectation to come out of the two year training 
sequence beyond the minimum standard for reaching full PBIS implementation fidelity.) 


Change recommendations regarding TFI completion at Spring training vs. end-of-year in-school walkthrough. 
The Minnesota PBIS Statewide Leadership Team has discussed and should consider making a final recommendation 
regarding eliminating the Spring training TFI in favor of encouraging all schools, especially those in the second year 
of training, to complete an end-of-year TFI walkthrough with an outside facilitator. (The RIPs and Wilder support 
this process by helping schools get matched up with other schools in their region who also want to complete a 
walkthrough TFI.) 


Consider reducing the emphasis on the other fidelity assessment tools in this report and on the PBIS-MN website 
in favor of increasing emphasis on the TFI, including expanding options for schools out of training to use Tiers 2 
and 3 of the TFI. 


For more information 


Wilder 
For more information about this report, contact 
Resea rc h Nicole MartinRogers at Wilder Research, 651-280-2682. 


Information. Insight. Impact. 
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! McIntosh, K., Predy, L., Uprety, G., Hume, A., Turri, M., and Mathews, S. (2013). Perceptions of Contextual Features Related to 


Implementation and Sustainability of School-wide Positive Behavior Support. Journal of Positive Interventions, 16(1), 31-43. 
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